{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ca2CpPPUvO-h"
      },
      "source": [
        "# **Google Colab Tutorial**\n",
        "\n",
        "Video: https://youtu.be/YmPF0jrWn6Y\n",
        "Should you have any question, contact TAs via <br/> ntu-ml-2022spring-ta@googlegroups.com\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xIN7RF4wjgHk"
      },
      "source": [
        "<p><img alt=\"Colaboratory logo\" height=\"45px\" src=\"/img/colab_favicon.ico\" align=\"left\" hspace=\"10px\" vspace=\"0px\"></p>\n",
        "\n",
        "<h1>What is Colaboratory?</h1>\n",
        "\n",
        "Colaboratory, or \"Colab\" for short, allows you to write and execute Python in your browser, with \n",
        "- Zero configuration required\n",
        "- Free access to GPUs\n",
        "- Easy sharing\n",
        "\n",
        "Whether you're a **student**, a **data scientist** or an **AI researcher**, Colab can make your work easier. Watch [Introduction to Colab](https://www.youtube.com/watch?v=inN8seMm7UI) to learn more, or just get started below!\n",
        "\n",
        "You can type python code in the code block, or use a leading exclamation mark ! to change the code block to bash environment to execute linux code."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IrAxlhp3VBMD"
      },
      "source": [
        "To utilize the free GPU provided by google, click on \"Runtime\"(執行階段) -> \"Change Runtime Type\"(變更執行階段類型). There are three options under \"Hardward Accelerator\"(硬體加速器), select \"GPU\". \n",
        "* Doing this will restart the session, so make sure you change to the desired runtime before executing any code.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CLUWxZKbvQpx",
        "outputId": "ad5ee344-67fa-4e6f-e99f-0d7a289517a5"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "execution_count": 12,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "import torch\n",
        "torch.cuda.is_available() # is GPU available\n",
        "# Outputs True if running with GPU"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1jpUD08lMiDa",
        "outputId": "ca16beaf-6200-46d7-d64c-b926eea744ba"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Fri Feb 18 10:43:31 2022       \n",
            "+-----------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 460.32.03    Driver Version: 460.32.03    CUDA Version: 11.2     |\n",
            "|-------------------------------+----------------------+----------------------+\n",
            "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                               |                      |               MIG M. |\n",
            "|===============================+======================+======================|\n",
            "|   0  Tesla K80           Off  | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   70C    P8    34W / 149W |      3MiB / 11441MiB |      0%      Default |\n",
            "|                               |                      |                  N/A |\n",
            "+-------------------------------+----------------------+----------------------+\n",
            "                                                                               \n",
            "+-----------------------------------------------------------------------------+\n",
            "| Processes:                                                                  |\n",
            "|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n",
            "|        ID   ID                                                   Usage      |\n",
            "|=============================================================================|\n",
            "|  No running processes found                                                 |\n",
            "+-----------------------------------------------------------------------------+\n"
          ]
        }
      ],
      "source": [
        "# check allocated GPU type\n",
        "!nvidia-smi"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EAM_tPQAELh0"
      },
      "source": [
        "**1. Download Files via google drive**\n",
        "\n",
        "  A file stored in Google Drive has the following sharing link：\n",
        "\n",
        "https://drive.google.com/open?id=1sUr1x-GhJ_80vIGzVGEqFUSDYfwV50YW\n",
        "  \n",
        "  The random string after \"open?id=\" is the **file_id** <br />\n",
        "![](https://i.imgur.com/77AeV88l.png)\n",
        "\n",
        "  It is possible to download the file via Colab knowing the **file_id**, using the following command.\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XztYEj0oD7J3",
        "outputId": "83ddde6e-d745-4a0f-bb8f-dab8243a1dca"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Downloading...\n",
            "From: https://drive.google.com/uc?id=1sUr1x-GhJ_80vIGzVGEqFUSDYfwV50YW\n",
            "To: /content/ML2022/pikachu.png\n",
            "\r  0% 0.00/204k [00:00<?, ?B/s]\r100% 204k/204k [00:00<00:00, 83.4MB/s]\n"
          ]
        }
      ],
      "source": [
        "# Download the file with file_id \"sUr1x-GhJ_80vIGzVGEqFUSDYfwV50YW\", and rename it to pikachu.png\n",
        "!gdown --id '1sUr1x-GhJ_80vIGzVGEqFUSDYfwV50YW' --output pikachu.png"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Gg3T23LXG-eL",
        "outputId": "0dd8a3ca-85d5-4bc0-d593-4a3ab1bbfc99"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "pikachu.png\n"
          ]
        }
      ],
      "source": [
        "# List all the files under the working directory\n",
        "!ls"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "38dcGQujOVWM"
      },
      "source": [
        "Exclamation mark (!) starts a new shell, does the operations, and then kills that shell, while percentage (%) affects the process associated with the notebook"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dOQxjfAZAsys"
      },
      "source": [
        "It can be seen that `pikachu.png` is saved the the current working directory. \n",
        "\n",
        "![](https://i.imgur.com/bonrOlgm.png)\n",
        "\n",
        "The working space is temporary, once you close the browser, the files will be gone.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "50uppJXgZwmW"
      },
      "source": [
        "Double click to view image\n",
        "\n",
        "![](https://i.imgur.com/DTywPzAm.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k_gmTo9NKtu9"
      },
      "source": [
        "**2. Mounting Google Drive**\n",
        "\n",
        "  One advantage of using google colab is that connection with other google services such as Google Drive is simple. By mounting google drive, the working files can be stored permanantly. After executing the following code block, your google drive will be mounted at `/content/drive`"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FyrDicHSYlb2"
      },
      "source": [
        "![](https://i.imgur.com/IbMf5Tg.png)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 347
        },
        "id": "BmvzTF5IJ6TL",
        "outputId": "84238034-a652-4f6d-dadf-421199e4f9a6"
      },
      "outputs": [
        {
          "ename": "MessageError",
          "evalue": "ignored",
          "output_type": "error",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mMessageError\u001b[0m                              Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-17-d5df0069828e>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mgoogle\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcolab\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mdrive\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mdrive\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmount\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'/content/drive'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/google/colab/drive.py\u001b[0m in \u001b[0;36mmount\u001b[0;34m(mountpoint, force_remount, timeout_ms, use_metadata_server)\u001b[0m\n\u001b[1;32m    116\u001b[0m       \u001b[0mtimeout_ms\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtimeout_ms\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    117\u001b[0m       \u001b[0muse_metadata_server\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0muse_metadata_server\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 118\u001b[0;31m       ephemeral=ephemeral)\n\u001b[0m\u001b[1;32m    119\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    120\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/google/colab/drive.py\u001b[0m in \u001b[0;36m_mount\u001b[0;34m(mountpoint, force_remount, timeout_ms, use_metadata_server, ephemeral)\u001b[0m\n\u001b[1;32m    139\u001b[0m   \u001b[0;32mif\u001b[0m \u001b[0mephemeral\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    140\u001b[0m     _message.blocking_request(\n\u001b[0;32m--> 141\u001b[0;31m         'request_auth', request={'authType': 'dfs_ephemeral'}, timeout_sec=None)\n\u001b[0m\u001b[1;32m    142\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    143\u001b[0m   \u001b[0mmountpoint\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_os\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpath\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexpanduser\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmountpoint\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/google/colab/_message.py\u001b[0m in \u001b[0;36mblocking_request\u001b[0;34m(request_type, request, timeout_sec, parent)\u001b[0m\n\u001b[1;32m    173\u001b[0m   request_id = send_request(\n\u001b[1;32m    174\u001b[0m       request_type, request, parent=parent, expect_reply=True)\n\u001b[0;32m--> 175\u001b[0;31m   \u001b[0;32mreturn\u001b[0m \u001b[0mread_reply_from_input\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrequest_id\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtimeout_sec\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/google/colab/_message.py\u001b[0m in \u001b[0;36mread_reply_from_input\u001b[0;34m(message_id, timeout_sec)\u001b[0m\n\u001b[1;32m    104\u001b[0m         reply.get('colab_msg_id') == message_id):\n\u001b[1;32m    105\u001b[0m       \u001b[0;32mif\u001b[0m \u001b[0;34m'error'\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mreply\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 106\u001b[0;31m         \u001b[0;32mraise\u001b[0m \u001b[0mMessageError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mreply\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'error'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    107\u001b[0m       \u001b[0;32mreturn\u001b[0m \u001b[0mreply\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'data'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    108\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mMessageError\u001b[0m: Error: credential propagation was unsuccessful"
          ]
        }
      ],
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AkmayCmGMD03"
      },
      "source": [
        "After mounting the drive, the content of the google drive will be mounted on a directory named `MyDrive`\n",
        "\n",
        "![](https://i.imgur.com/jDtI10Cm.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UhKhwipoMvXF"
      },
      "source": [
        "After mounting the drive, all the changes will be synced with the google drive.\n",
        "Since models could be quite large, make sure that your google drive has enough space."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UT0TEPRS7KF6",
        "outputId": "291edb11-1341-405b-ec74-d915976ff90c"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[Errno 2] No such file or directory: '/content/drive/MyDrive'\n",
            "/content\n",
            "/content/ML2022\n"
          ]
        }
      ],
      "source": [
        "%cd /content/drive/MyDrive \n",
        "#change directory to google drive\n",
        "!mkdir ML2022 #make a directory named ML2022\n",
        "%cd ./ML2022 \n",
        "#change directory to ML2022"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Oj13Q58QerAx"
      },
      "source": [
        "Use bash command pwd to output the current directory"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-S8l1-ReepkS",
        "outputId": "051dc816-95b9-4922-bab9-b0c40b3d7909"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "/content/ML2022\n"
          ]
        }
      ],
      "source": [
        "!pwd #output the current directory"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qSSvrDaBiDrP"
      },
      "source": [
        "Repeat the downloading process, this time, the file will be stored permanently in your google drive."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "b39YMYicASvP",
        "outputId": "0b08768d-2154-4a6a-d03c-e04f966f87ed"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Downloading...\n",
            "From: https://drive.google.com/uc?id=1sUr1x-GhJ_80vIGzVGEqFUSDYfwV50YW\n",
            "To: /content/ML2022/pikachu.png\n",
            "\r  0% 0.00/204k [00:00<?, ?B/s]\r100% 204k/204k [00:00<00:00, 82.1MB/s]\n"
          ]
        }
      ],
      "source": [
        "!gdown --id '1sUr1x-GhJ_80vIGzVGEqFUSDYfwV50YW' --output pikachu.png"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BRTF7yxTY-aE"
      },
      "source": [
        "Check the file structure\n",
        "\n",
        "![](https://i.imgur.com/DbligmOt.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D0URgikZXl5I"
      },
      "source": [
        "For all the homeworks, the data can be downloaded and stored similar as demonstrated in this notebook. "
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [],
      "name": "Google Colab Tutorial 2022",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
